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1 Administrative Details 

• Signup online for scribing. 

2 Review of Lecture 1 

All of the following are covered in detail in the notes for Lecture 1: 

• The definition of Lq, specifically that Lq = Dg — Aq, where Dq is a diagonal matrix of degrees and 
A G is the adjacency matrix of graph G. 

• The action of Lq on a vector x, namely that 

[1(3^ = deg(i) (xt — average of x on neighbors of i) 

• The eigenvalues of Lq are Ai < • • • < A„ with corresponding eigenvectors vi,...,v n . The first and the 
most trivial eigenvector is V\ = 1 with an eigenvalue Xi = 0. We are mostly interested in v 2 and A 2 . 

3 Properties of the Laplacian 

3.1 Some simple properties 

Lemma 1 (Edge Union) If G and H are two graphs on the same vertex set with disjoint edge sets, 

Lguh = Lg + Lh (additivity) (1) 

Lemma 2 (Isolated Vertices) If a vertex i € G is isolated, then the corresponding row and column of the 
Laplacian are zero, i.e. [Lq]^ = [Lc]j i = for all j. 

Lemma 3 (Disjoint Union) These together imply that the Laplacian of the disjoint union of G and H is 
direct sum of Lq and Lh, i-e.: 

L G 

'H 



Lguh = Lg®L h = ( Q L r ) (2) 



Proof Consider the graph G II v(H) = (Vq U Vh,Eg), namely the graph consisting of G along with the 
vertex set of H as disjoint vertices. Define v(G) II H similarly. By the second remark, 

Lguv(h) = ( L q q ^ and L v(G)UH = ^ jj L H \ 

By definition, GUH=(GU v(H)) U (v(G) II H), and so by the first remark: 

( L G 

Lguh - L G ® L H - y Q ^ 
This implies the Laplacian is the direct sum of the Laplacians of the connected components. Thus, 
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Theorem 4 (Disjoint Union Spectrum) If Lq has eigenvectors vi, . . . , v n with eigenvalues Ai, . . . , X n , 
and Lh has eigenvectors u>i, . . . , w n with eigenvalues jjb\, . . . , jjb n , then Lquh has eigenvectors: 



vi © o, . . . , v n © o, o e w 1} . . . , o e w n 

Ai, . . . , A„, fix, . . . , fj, n . 



with corresponding eigenvalues: 

Proof By the previous lemma, 

Lguh* (vi®0) = 

Thus vi © is an eigenvector of Lguh with eigenvalue Ai. The rest follow by symmetry. 



Ai^i 



L G 

L H ) \ } \ 



3.2 The Laplacian of an edge 

Definition 5 Let L e be the Laplacian of the graph on n vertices consisting of just the edge e. 
Example 1 If e is the edge (vi,v 2 ), then 



( 1 -1 
-110 


V o oo 



o \ 



o J 



(3) 



By additivity, this lets us write: 

L G - ]T L e 

This will allow us to prove a number of facts about the Laplacian by proving them for one edge and 
adding them up. The more general technique, which we'll use more later, is to bound Laplacians by adding 
matrices of substructures. 



So for an edge e, 



Note that: 



1 -1 
-1 1 



[zeros] . 



1 -1 
-1 1 



and so ^ — 73 ) is an eigenvector with eigenvalue 2. This decomposition implies: 

x T L e x = ( xi x 2 ) ( j ( 1 -l)( Xl )=( Xl - x 2 f ■ 



Remark The Laplacian is a quadratic form, specifically: 



X T LqX — X T (^2, Le)x = ^ X T L e X = (Xi~ XjY 

e<£E e£E (i,j)eE 



(4) 



(5) 



This implies that L is positive semidefinite. 
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3.3 Review of Positive Semidefiniteness 

Definition 6 A symmetric matrix M is positive semidefinite (PSD) z/Vx € R™, 

x T Mx > 0. 

M is positive definite (PD) if the inequality is strict Vi^ 0. 

Lemma 7 M is PSD iff all eigenvalues Aj > 0. Similarly M is PD iff all eigenvalues Aj > 0. 

Proof Let's consider the matrix M in its eigcnbasis, that is M — Q T AQ. Clearly, y T Ay = Xtyf > 
for all y € R™ iff Aj > for all i. Similar for PD matrix. ■ 

Lemma 8 (PSD Matrix Decomposition) M is PSD iff there exists a matrix A such that 

M = A T A. (6) 
Note that A can be (n x k) for any k, and that it need not be square. Importantly, note that A is not unique. 
Proof 

(=>) If M is positive semidefinite, recall that M can be diagonalized as 

M - Q T AQ, 

thus 

M = Q T K 1/2 K 1/2 Q = (a^qY (k 1 ' 2 Q) , 
where A 1 / 2 has \/Aj on the diagonal. 
(<=) If M = A T A, then 

x T Mx = x T A T Ax = (Axf(Ax) 
Letting y = (Ax) € R fc , we see that: 

x T Mx = y T y = \\y\\ 2 >0. ■ 



3.4 Factoring the Laplacian 

We know from the previous section that we can factor L as A T A using eigenvectors, but there also exists a 
much nicer factorization which we will show here. 

Definition 9 Let m be the number of edges and n be the number of vertices. Then the incidence matrix 
V = Vg is the m x n matrix given by: 

{1 if e = (v, w) and v < w 
— 1 if e = (v,w) and v > w (7) 
otherwise. 
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Example 2 The Laplacian and the Incidence matrix of the graph G= 
3 4 

\ -1 1 / \iuu i/ 

2 

Lemma 10 L G = V T V. 

Proof Observe that [V T V] ... = (ith column of V)-(jth column of V) = £ e (We,^) ([V]^] This gives 
three cases: 

• When i = j, 

e e incident to Uj 

• When i j and no edge exists between v% and u,, 

[^]« = E(m.„)(m.,,,,) = o 

e 

as every edge is non-incident to at least one of , «j . 

• When i ^ j and exists an edge e' between n and Vj , 

W^ii = E (Me.,,) ([VlJ = ([*],,„) ([V] e> J = "I- ■ 

e 

Corollary 11 iVoie i/iai 

x t Lqx — ||Vx|| 2 = (xi — Xj) 2 , 

(i,j)eE 

This gives another proof that L is PSD. 

3.5 Dimension of the Null Space 

Theorem 12 If G is connected, the null space is 1 -dimensional and spanned by the vector 1. 
Proof Let x G null(L), i.e. Lqx = 0. This implies 

x T L G x~ (xi — Xj) 2 = 0. 

Thus, Xi = xj for every £ E. As G is connected, this means that all Xi are equal. Thus every member 
of the null space is a multiple of 1. ■ 

Corollary 13 If G is connected, A2 > 0. 

Corollary 14 The dimension of the null space of Lq is exactly the number of connected components of G. 
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4 Spectra of Some Common Graphs 

We compute the spectra of some graphs: 

Lemma 15 (Complete graph) The Laplacian for the complete graph K n on n vertices has eigenvalue 
with multiplicity 1 and eigenvalue n with multiplicity n — 1 and associated eigenspace {x\x ■ 1 = 0}. 

Proof By corollary 13, we conclude that eigenvalue has multiplicity 1. Now, take any vector v which is 
orthogonal to 1 and consider [Lx n v)i. Note that this value is equal to 

(n — l)vi — Vj = nvi — Vj = nV{ 

Hence any vector v which is orthogonal to 1 is an eigenvector with an eigenvalue n. M 

Lemma 16 (Ring graph) The Laplacian for the ring graph R n on n vertices has eigenvectors 

Xk{u) = sin(27rfcu/n), and 
Vk{u) — cos{2irku/n) 

for < k < n/2. Both Xk and yk have eigenvalue 2 — 2cos(2irk/n). Note that, xo — should be ignored and 
Uq is 1, and when n is even x n / 2 — should be ignored and we only have y n /2- 

Proof . The best way to see is to plot the graph on the circle using these vectors as coordinates. Below is 
the plot for a k = 3. 
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Just consider vertex 1. Keep in mind that sin(2a;) = 2 sin(x) cos(.t). Then, 

[Lx h ]x = 2x k {\) - x k {Q) - x k {2) 

= 2sin(27rfc/n) -0-sin(27rfc2/n) 

= 2sin(27rfc/n) — 2 sin(27r/c/n) cos(2wk/n) 

= (2 - 2cos(27rA:/n))sin(27rfc/n) 

= (2 - 2cos(2irk/n))x k (l) 

m 

Note that this shows that x(u) = Ji( e 27 ™( fcu + C )/™) j s an eigenvector for any k € Z, c e C. 

Lemma 17 (Path graph) The Laplacian for the path graph P n on n vertices has the same eigenvalues as 
i?2n o,nd eigenvectors 

Vk{u) — sin(nku/n + 7r/2n) 

for < k < n 
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Proof 



12 3 R10 
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We will realize P n as quotient of i?2n- Suppose z was an eigenvector of Lp 2n in which Zj — z 2n -i-i for 
< i < n. Take the first n components of z and call this vector v. Note that for < i < n: 

[Lp n v]i — 2(vj — neighbors of i in P n ) 
= 2(zi — neighbors of i in R% n ) 

— (zi — neighbors of i in R,2 n ) + fan— i-i ~ neighbors of (2n — i — 1) in J?2n) 

= Xzi 
= XVi 

Now consider the case when i = 0. 

[£p n t>]o = «q-«i 

= 2u - wi + v 

= 2z - zi + z Q 

— 2z — Zi + Z-2n-l 

= Az 
= Aw 

Hence, v is an eigenvector of Lp n - Now we show that such v exists, that is, there exists eigenvector z of 
L/? 2n in which Zi — Z2 n -i—i for < i < n. Take z, 

z k(u) = sin(7rfcu/n + Tr/2n) 

= smijrku/n) cos(7r/2n) + cos^irku/n) sin(7r/2n) 
= Xk(u) cos(7r/2n) + yf. sin(7r/2n) 

We see that z^ is in the span of and y^. Hence, zj, is an eigenvector of Lp 2n with an eigenvalue 
2 — 2 cos(27rfc/n) by lemma 16. Check that Zk satisfies Zk(i) — Zk(2n — 1 — i) ■ 

4.1 Graph Products 

The next natural example is the grid graph, which will follow from general theory about product graphs. 
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Definition 18 Let G = (V, E), and H — (W, F). The product graph G x H has vertex setVxW and edge 
set: 

((«!, w), (v 2 , w)) , V(«i, v 2 ) S E, w e W and 
((v,w 1 ),(v,w 2 )) , V{w 1 ,w 2 ) e F,v eV. 

Example 3 P n x P m = G n , m - We see that the vertices of P n x P m are: 

(vi,wi) {vi,w 2 ) ■■■ (vi,w m ) 
(v 2 ,wi) (v 2 ,w 2 ) ■■■ {v 2 ,w m ) 
v(G n ^ n ) = < . 

(v n ,wi) (v n ,w 2 ) ■■■ (v 

The vertices are written in the above layout because it makes the edges intuitive. The edges are: 

• For a fixed w, i.e. a column in the above layout, a copy of P n . 

• For a fixed v, i.e. a row in the above layout, a copy of P m . 

Thus P n x P m = G n , m , which is to say the product of two path graphs is the grid graph. 

Theorem 19 (Graph Products) If Lq has eigenvectors v\, . . . ,v n with eigenvalues Ai, . . . , \ n , and Lh 

has eigenvectors Wi, . . . ,Wk with eigenvalues fj,i, . . . ,Hk, then L GxH has, for all 1 < i < n, 1 < j < k, an 
eigenvector: 

Zij(v,w) = x l (v)y j (w) 

of eigenvalue Xi + fj,j . 

Note importantly that eigenvalues add here, they do not multiply. 

Proof Let A m be the graph with m isolated vertices. We can then decompose the product as: 

Gx H=(Gx A k )U(A n x H), 

i.e. the edge union of k disjoint copies of G and n disjoint copies of H, exactly as in the definition. By 
Lemmas 1 and 3 we have 

Lgxh = LcxA k + La„xh = L G ® Jfc + 7„ ® L H 

Consider Zij = Xi ® yj as above for a fixed i and j, we see that: 

LcxHZij = {L G ®Ik){x i ®y j ) + (I n ®L H )(xi®y j ) 
= (XiXi ® yj) + (xi ® fijyj) 
= ( x i + Mj) ( x i ®Vj) = ( x i + N) Z H- ■ 

Corollary 20 G n . m has eigenvectors and eigenvalues completely determined by those of P n and P m as above. 

5 Why is this called the Laplacian? 

It turns out that the graph Laplacian is very naturally related to the continuous Laplacian. 

• In 1 dimension, the continuous Laplacian is 

• In 2 dimensions, the continuous Laplacian is V/ = ^Js + 
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5.1 Discretizing Derivatives, Id case 

Consider a Id function / : K — > K, which we wish to discretize at the points (. . . , k — 2, k—1, k, fc+1, k+2, . . .): 

f (k-2) f(k-l) f(k) f(k+l) f(k+ l) 

We approximate the first derivative at the line midpoints, up to scaling, by taking the differences 
between the values at the adjacent points: 



f(k-l]-f(k-2) f(k)-f(k-l) f(k+l)-f(k) f(k+2)-f(k+l) 




The discrete first derivative of / is a function on edges, and is, up to scaling Vp n /, the incidence matrix 

d 2 f 

of P n defined earlier. In order to compute the second derivative at the original points, again up to 
scaling, we take the differences of the adjacent midpoints at the vertices: 

• | • | • | • | ■ 

[f(k)+f(k-2)] [f(k+l)+f{k-l)] [f(k)+f(k-2)] 
-2f(k-l) -2f(k) -2f(k-l) 

The discrete second derivative of / is thus, up to scaling, —Lp n f. 

5.2 Discretizing Derivatives, 2d case 

Here we discretize / : M. 2 — > R on a grid: 



if(k-l,k+l) i f(k,k+l) i f(k+l,k+l) 



f(k-l,k) f(k,k) f(k+l,k) 



f(k-l.k-l) ..f(k,k-l) f(k+l,k-l) 



To compute the discrete derivative in the x and y directions, we'll just look at a little piece. On the 
horizontal edges, we approximate 4^ up to scaling, and do likewise on the vertical edges with Jjj: 
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f(k,k+l)-f(k,k) 



T f(k+ 



f(k+l,k)-f(k,k) 
f(k,k)-f(k,k-l) 



Again, the discrete derivative of / is a function on edges. When we consider the concatenation of the 
two discretization of the directional derivatives, we see that the discretization of the gradient, up to scaling, 
is Vg„ m f. Finally we use this to compute the discretized Laplacian, up to scaling, and get: 



f (k,k+ 1 )+f (k,k- 1 )+f (k+ 1 ,k)+f (k- 1 ,k) 
-4 f(k,k) 



Thus the discretized Laplacian in two dimensions of / is — Lg„ m /. 



5.3 A Note on Fourier Coefficients 

We observed that paths and rings had eigenvectors that looked like Fourier coefficients. In the continuous 
case: 



d 2 sin (kx + c) 
dx 2 



— k 2 sm{kx + c) 

LIX~ 

d 2 cos(kx + c) 2 



dx 2 



k cos (kx + c) 



Thus sin(kx + c) and cos(fcx + c) are eigenfunctions of the £^ operator, i.e. the Id Laplacian, both with 
eigenvalue k 2 . 

6 Bounding Laplacian Eigenvalues 

Lemma 21 (Sum of the eigenvalues) Given an n-vertex graph G with degrees di, where d max = max^ di, 
and Laplacian Lq with eigenvalues Aj, 

^ Aj = ^ di < d max n (8) 

i i 

Proof The first two expressions are both the trace, the upper bound is trivial. ■ 
Lemma 22 (Bounds on A2 and A„) Given Xi and di as above, 

A 2 < ^4 ( Q ) 
n — 1 
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(10) 



Proof By the previous slide and the fact that Ai — 0, we get Ef=2 A» = Ej As A 2 < • • • < A„, the 
bounds follow immediately. ■ 



7 Bounding A2 and A max 

Theorem 23 (Courant-Fischer Formula) For any n x n symmetric matrix A, 



Ai 



A 2 = 



A„ 



min x T Ax 
lkll=i 



min x T Ar 
11*11=1 



max x T Ax 
lkll=i 



. x T Ax 

mm — = — 
k^o x J x 

. x T Ax 

mm — = — 

x^O x J X 

x T Ax 
max — Tf, — 

x^O X 1 X 



(11) 



Proof We consider the diagonalization A = Q T AQ. As seen earlier, x T Ax — (Qx) T A(Qx). As Q is 
orthogonal, we also have \\Qx\\ = x. Thus it suffices to consider diagonal matrices. Moreover, all of the 
equalities on the right follow immediately from linearity Thus we need to consider, for ||x|| = 1: 



x T Ax = (xi • • • x„) 



V 



\ 



A„ J 



( X! \ 



\ x n J 



- Va x 2 - ^ AiX ' 2 
^ 1 1 E*l ■ 



We compute the gradient and find: 

[V, (^ T Ax)] i 



2AiXi 



2A 4 x3 



= 2A 4 (x, -x?) 



(E- 2 ) 2 

thus all extremal values occur when one Xj = 1 and the rest are 0. The identities follow immediately. 
Corollary 24 (Rayleigh Quotient) Letting G — (V,E) be a graph with Laplacian Lq, 



Ai 
A 2 



= 



. x LqX 

mm — = 



x^O 



mm 

£x=o 
x^O 



E(i,j)£B x j) 



E 



;ev 2:2 



(12) 



X LqX 



max ■ 

k^o x J x 



max 



E(i,j)£E ( X « X j) 



E 



The Rayleigh Quotient is a useful tool for bounding graph spectra. Whereas before we had to consider 
all possible vectors x, now in order to get an upper bound on A 2 we need only produce a vector with small 
Rayleigh quotient. Likewise to get a lower bound on A max we need only to find a vector with large Rayleigh 
quotient. 

Examples next lecture! 
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